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Description 

[0001] The present invention relates to a method of 
transmitting data streams between a sending host and 
a receiving host using a transport protocol such as TCP. 
[0002] In network communications between a send- 
ing host and a receiving host, the transmission of data 
packets is mostly done on a best-effort basis. In the 
communication link, the available bandwidth and the 
current link conditions vary and hence, the end hosts 
must be provided with mechanisms to probe and adjust 
themselves to the current link conditions. 
[0003] With the development of 3rd generation net- 
works, so-called streaming applications are gaining im- 
portance. Streaming applications are understood as a 
continuous stream of data, e.g. video or audio data 
which is typically sensitive to time delays. In order that 
the user will be able to receive multimedia data streams 
on a portable receiver, for example a mobile phone, sev- 
eral requirements on the transport mechanism will be 
imposed in order that the delivered data meets with the 
resulting time constraints. These constraints are difficult 
to meet in a best-effort environment, which is subject to 
bandwidth variations and delays. 
[0004] To cope with the problem of bandwidth adap- 
tion, it is common praxis to make several data streams 
with different encoding rates but same content available 
at the sending host. According to the available band- 
width, it is then switched between the data streams in 
order to adjust the rate to the available bandwidth. In 
other words, multi-rate content streams at the sending 
host are transmitted and switching operations between 
streams are performed depending on the available 
bandwidth. 

[0005] There are some conventional approaches to 
TCP streaming that implement a client buffer manage- 
ment, but there are various constraints which are de- 
sired to overcome. First of all, the approach to imple- 
ment a client buffer management should not require 
modifying the original encoded data or the TCP protocol 
in any way. Further, the need for additional application 
messages should not arise, otherwise, the method be- 
comes difficult to implement. Finally, it should be avoid- 
ed to discard data packets which leads to a loss of data 
and is therefore not an optimal solution. 
[0006] In case of multi-media streaming using the 
TCP protocol, the specific problem lies in achieving a 
rate match between the bit rate which the TCP flow con- 
trol mechanism is able to get out of the current trans- 
mission link and the rate at which the client application 
requests data. Assuming that multi-rate content is avail- 
able at the server, the problem is reduced to how to 
make a decision to switch this stream and to which 
stream it should be switched. 

[0007] Therefore, the problem underlying the present 
invention is to provide a method of transmitting data 
streams between a sending host and a receiving host, 
wherein said rate match is achieved in a simple manner 



and which is easy to implement. 

[0008] This object is solved by a method comprising 

the steps as set forth in claim 1 . 

[0009] The idea underlying the invention is to monitor 

5 the occupancy state of the application layer buffer at the 
receiving host and to signal stream control data to the 
sending host depending on the monitored state of the 
application layer buffer. In this way, a close monitoring 
of the application layer buffer can be performed. 

10 [0010] According to a preferred embodiment, the 
stream control data commands the sending host to limit 
or change the encoding rate of the data stream. 
[0011] According to a further advantageous embodi- 
ment of the method, the stream control data signals to 

'5 the sending host to switch from one data stream to an- 
other data stream having the same content but a differ- 
ent encoding rate. Consequently, the buffer monitoring 
operation triggers the issuing of appropriate requests to 
switch or to prepare the switch procedure between 

20 streams. 

[0012] Preferably, the step of monitoring the occupan- 
cy state of the application layer buffer comprises to com- 
pare the bit rate at which the receiving host delivers new 
data to the application layer buffer with the rate at which 
25 data is requested by the application. 

[0013] According to a further preferred embodiment, 
the actual occupancy state is compared against a plu- 
rality of occupancy thresholds, which is an easily imple- 
mented procedure. Upon exceeding or falling below cer- 
30 tain buffer occupancy thresholds, appropriate action will 
be carried out to ensure a seamless stream switch. 
[0014] The present invention will be more fully under- 
stood from the following detailed description with refer- 
ence to the accompanying drawings which show: 

35 

Figure 1: the basic architecture of a sending host 
and a receiving host connected by a com- 
munications network in which the method 
subject to the present invention can be 
40 performed, 

Figure 2: a block diagram of a server architecture as 
the sending host in which the method of 
the present invention is performed, 

45 

Figure 3: a block diagram of a client architecture, 
and 

Figure 4: a schematic diagram of an application lay- 
50 er buffer illustrating several occupancy 

states. 

[0015] Figure 1 illustrates the basic architecture of a 
communications network for connecting a video server 
55 as a sending host to a video client acting as a receiving 
host. The communications network is, for example, a 
UMTS core network, an external network, such as the 
internet or an internal intranet over which the server and 
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the client communicate by means of a transport proto- 
col. As transport protocol TCP is selected as an exam- 
ple, but it is clearto a skilled person that any other trans- 
port protocol for transmitting data can be employed. 
[0016] In a more general way, it will now be explained 
how TCP handles the exchange of data between the 
server and the client. 

[0017] First of all, the video server needs to transfer 
its data to the protocol and tell it to send same to the 
client. To progressively read and send data, the protocol 
uses a kind of buffer with added functionalities, a so- 
called TCP socket. Generally, sockets are unequivocal- 
ly identified by the IP addresses of the correspondents, 
the protocols and ports which are used. Once a connec- 
tion between two sockets is established, the communi- 
cation can be controlled by use of interface calls, which, 
for example may comprise "create a socket", "bind a 
socket to a transport address", "accept" or "close a con- 
nection", "send", "receive" or "write" data between the 
sender and the receiver. Wide-spread socket implemen- 
tations are UNIX BSD 4.2 and Windows Winsock 2.0, 
where equivalent interface calls are implemented. 
[0018] Each interface call has parameters, e.g.forthe 
"send" call, parameters of relevance are the socket de- 
scriptor, the identifier of the data to be sent and the 
length of the data stream. Once a data stream is passed 
in one call, no single bit of this data stream can be mod- 
ified. The only way to abort transfer of this data stream 
is to break down the connection. 
[001 9] Nevertheless, in order to close the connection, 
the application must explicitly issue a "close" call. Inter- 
face calls can be queued as, for example, in FTP. When 
asked for a file, the FTP server usually issues two inter- 
face calls for this transmission, namely a "send" call in- 
dicating the socket, the identifier of the data stream and 
the length thereof and a "close" call identifying the sock- 
et. The two interface calls are put into the queue and 
when the "send" call is completed, the connection is ir- 
reversibly broken down because the "close" call was 
next in the queue. 

[0020] Alternatively, for example, in case of TELNET, 
the connection is established and the socket is set to 
wait for incoming data, i.e. commands typed on the key- 
board. 

[0021] Of course, the application can be configured 
such that the server only issues a send call and waits 
for the client to close. However, this is uncommon. 
[0022] Additionally, it is assumed that the TCP proto- 
col provides a means for asynchronously signaling cer- 
tain information about events in the socket. Often in the 
specification the information will be an error message. 
In other cases, there will be information relation to the 
completion of processing a SEND or RECEIVE or an- 
other interface call. 

[0023] Referring again to Figure 1 , the video server 
provides, e.g. an MPEG 4 compressed data stream, i. 
e. a given video content, which is made available to the 
TCP socket in the form of mono-rate or multi-rate 



streams. Multi-rate streams are defined forthe purpose 
of this and the following description, as one or several 
data streams with the same video content at different 
encoding rates. The MPEG data streams are forwarded 
5 to the TCP socket. Subsequently, by means of an IP pro- 
tocol, data is transmitted to the video client. A stream 
control protocol, e.g. RTSP, should be used to perform 
such operations as PLAY, PAUSE or STOP the trans- 
mission. 

10 [0024] With reference to Figure 2, which shows the 
basic architecture of a video server 200, at the MPEG 
server application 21 0, video content is made available 
in a buffer 211 at different coding rates. A switch 212, 
which receives stream control RTSP commands from 

15 RTSP block 220, such as "Play", "Pause", "Tear down" 
or "Change rate", selects one of the data streams which 
is forwarded to the TCP socket 230 inform of an MPEG 
file. 

[0025] It is clear to a skilled person that other stream 

20 control protocols than RTSP could be used atthe server 
and the client. The stream control protocol typically in- 
structs to perform play, forwarding, rewinding, pause or 
stop functions. Additionally, if multi-rate content is avail- 
able, switch to a higher or lower encoding rate can be 

25 instructed. 

[0026] The server application forwards data seg- 
ments of variable length to the TCP socket 230. The 
TCP socket forms packet data units (PDUs) for trans- 
mission to the client application. 

30 [0027] TCP flow control continuously probes the net- 
work for available bandwidth by allowing one additional 
packetto be in flight every roundtrip time (RTT). Packets 
in flight are those which are sent without waiting for the 
respective acknowledgement. This continuous probing 

35 for bandwidth is unaware that the delivered bit rate of 
the TCP protocol might be too high for the receiving cli- 
ent application, i.e. available link bandwidth is 128 Kbps 
while the stream of data is 64 Kbps encoded video data. 
In this case, the application layer buffer would overflow 

40 and packet losses would take place. Similarly, the avail- 
able bandwidth of the communication link might not be 
enough to transmit the data with the encoding rate that 
the client application would prefer. When such mis- 
matches occur, packet losses take place. As a result, 

45 congestion control is invoked and the TCP throughput 
reduced. With the present invention, these rate mis- 
matches are avoided. 

[0028] The basic architecture of a TCP streaming cli- 
ent 300 is illustrated in Figure 3. The client 330 typically 

50 consists of a TCP socket 31 0, an application layer buffer 
320, a buffer manager 330 forthe application layer buff- 
er and a display 350 for visualization purposes. 
[0029] The client 300 processes the data segments 
of variable length as they come. In response thereto, it 

55 sends ACKs and performs flow control in order to pre- 
vent an overflow of the TCP socket 31 0 while extracting 
the maximum bit rate possible from the link. To do this, 
the TCP socket indicates to the server, with each ACK, 
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how much data it can receive until the socket is full using 
a window field in the header of the TCP ACK messages. 
If needed, the TCP client signals a zero-window telling 
the server to enter the "Persist" mode wherein no data 
is sent until the client allows to do so. Meanwhile the 
connection is maintained. The Persist mode and the sig- 
naling of a window field are standard functionalities of 
the TCP protocol. When the TCP socket has capacity 
for more data, it signals this to the server. Streaming 
control commands are signalled to the server by means 
of RTSP commands. In this manner, the frequency of 
sending rate reduction events can be decreased, and 
the delay and packet losses involved in abrupt connec- 
tion teardowns are avoided. 

[0030] The application layer buffer 320, also called 
"play-out buffer" receives data from the socket and 
transmits same to the MPEG client application 340. The 
manager 330 for the application layer buffer 320 re- 
ceives a frame requestfrom the MPEG client application 
and issues a socket request upon considering the state 
of the application layer buffer. The buffer manager 320 
therefore insures that the bit rate with which the TCP 
socket 310 can deliver the new data to the application 
layer buffer 320 matches the bit rate at which the appli- 
cation layer buffer is emptied to the MPEG client appli- 
cation 340. 

[0031] The buffer manager 330 is the one in charge 
of monitoring the fill-up status of the buffer. The TCP 
client socket 310 delivers its data to the buffer as long 
as there is capacity. 

[0032] Let's now assume a loss-free (no bit-error loss- 
es) and finite bandwidth link. In the initial state the buffer 
occupancy has an optimum value, the buffer is in "OK" 
state, say between the maximum and minimum occu- 
pancy thresholds. At every given instant the manager 
330 may observe one of the four different buffer states 
as illustrated in Figure 4. 

• "Full": some maximum occupancy threshold (Max. 
Fill-up_Level + Receiver_Socket_Size <= Maxi- 
mum Buffer Size) is surpassed or what's equivalent: 
the buffer manager is not able to accept the data 
from the TCP socket as fast as the socket is cur- 
rently able to deliver, because the client application 
display rate is smaller than the available link band- 
width. This causes the TCP client to progressively 
decrease the advertised window until a zero-win- 
dow is sent. Upon this, the server application enters 
the "Persist" Mode. 

In case multi-rate content is available at the 
server, the buffer manager additionally instructs the 
TCP server, via RTSP or another used stream con- 
trol protocol, to send another stream with higher en- 
coding rate. If not, the solution would be to limit the 
display rate, by just limiting the rate of the data that 
comes from the TCP socket. 

• "Empty": some minimum occupancy threshold is 
not reached. This means that the buffer manager 



consumes the data in the buffer faster that the TCP 
socket can deliver it. 

If multi-rate content is available the solution is, 
via RTSP, to stop taking data from the current en- 

5 coding rate and play from another stream encoded 
at a lower bit rate. If not, the solution is to stop the 
transmission, if the user desires. 
• "Incipient Switch Downwards (Upwards)": these 
thresholds are placed between both "Full" and 

10 "Empty" thresholds. The fill-up rate of the buffer de- 
creases ( or increases), the buffer empties ( or fills 
up) slowly. Action for the client to take: signal the 
server to initiate adequate actions to be prepared 
for a possible stream switch event. 

'5 • "OK": the buffer occupancy is optimum. It stays 
within the two "Incipient Switch Downwards (Up- 
wards)" thresholds because both available band- 
width and display rate match . 

20 [0033] This scheme can be easily generalized to the 
case of packet losses. The difference between "OK" and 
"Full (Empty)" fill-up levels shall account for TCP's in- 
stantaneous throughput variations. While the difference 
between the "Incipient Switch Downwards (Upwards)" 

25 and "Full (Empty)" fill-up levels shall avoid false switch 
decisions. 

[0034] The above-described mechanism allows the 
client application to control and, if required, limit the 
sending rate of a server stream application. The protocol 

30 used by both applications is TCP, which includes flow 
and congestion control algorithms to avoid buffer over- 
flow and is able to react appropriately in case of con- 
gestion. The described buffer monitoring operation is 
specifically useful when streaming real-time multimedia 

35 data over the TCP protocol and when the server is pro- 
vided with multi-rate contentthat is the same multimedia 
content at different encoding rates. This scheme allows 
loss-free or near-loss-free switching between streams. 
[0035] The above-described mechanism allows a cli- 

40 ent-driven sending rate limitation and/or rate adaption 
in case that multi-rate content is available to the server 
application. 



1. A method of transmitting data streams between a 
sending host and a receiving host using a transport 
protocol, comprising the steps of: 

50 

monitoring the occupancy state of a client ap- 
plication layer buffer of the receiving host, and 

signalling stream control data through the 
55 sending host depending on the monitored state 

of the application layer buffer. 

2. The method according to claim 1, wherein the 
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stream control data commands the sending host to 
limit or change the encoding rate of the data stream. 

3. The method according to claim 1 or 2, wherein the 
data stream is made available at the sending host 5 
in form of multi-rate streams of identical content. 

4. The method according to claim 3, wherein the 
stream control data signals to the sending host to 
switch from one data stream to another data stream 10 
having the same content but a different encoding 
rate. 

5. The method according to one of claims 1 -4, wherein 
the step of monitoring the occupancy state of the 
application layer buffer comprises comparing the bit 
rate at which the receiving host delivers new data 
to the application layer buffer with the rate with 
which data is requested by the application. 

6. The method according to one of claims 1 -5, wherein 
the step of monitoring the occupancy state of the 
application layer buffer comprises the step of com- 
paring the actual occupancy state against a plurality 
of occupancy thresholds. 

7. The method according to one of claims 1 -6, wherein 
the data streams are real-time multimedia data. 

8. The method according to one of claims 1 -7, wherein 
the transport protocol is TCP which includes a flow 
and congestion control algorithm which probes the 
communication link between the sending host and 
the receiving host for the available bandwidth. 

9. The method according to one of claims 1 -8, wherein 
the data streams are temporarily stored in an inter- 
nal buffer at the receiving host. 

1 0. The method according to one of claims 1 -9, wherein 40 
the stream control data is in accordance with the 
RTSP protocol. 
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